Papers with document-level representations
Text Generation from Discourse Representation Structures (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models to generate text from formal meaning representations based on Discourse Representation Structures (DRSs) . |
| Approach: | They propose neural models to generate text from formal meaning representations based on Discourse Representation Structures (DRSs). |
| Outcome: | The proposed model achieves competitive performance on the GMB benchmark against several strong baselines. |
MIReAD: Simple Method for Learning High-quality Representations from Scientific Documents (2023.acl-short)
Copied to clipboard
| Challenge: | Pretrained language models can learn rich textual representations, but they cannot provide powerful document-level representations for scientific articles. |
| Approach: | They propose a transformer-based method that learns semantically meaningful representations from scientific papers by fine-tuning transformer models to predict the target journal class based on the abstract. |
| Outcome: | The proposed method outperforms six existing models for representation learning on scientific documents across four evaluation standards. |
Are the Best Multilingual Document Embeddings simply Based on Sentence Embeddings? (2023.findings-eacl)
Copied to clipboard
| Challenge: | obtaining document embeddings at document level is challenging due to computational requirements and lack of appropriate data. |
| Approach: | They compare methods to produce document-level representations from sentences based on LASER, LaBSE, and Sentence BERT pre-trained multilingual models. |
| Outcome: | The proposed methods produce document-level representations from sentences in 8 languages . the results show that a clever combination of sentence embeddings is usually better than encoding the full document as a single unit. |
Recurrent Attention Networks for Long-text Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to encoding long documents using self-attention have been limited by quadratic computational complexities and limited application in long text processing. |
| Approach: | They propose a long-document encoding model that allows the recurrent operation of self-attention. |
| Outcome: | The proposed model extracts global semantics in token-level and document-level representations, making it inherently compatible with both sequential and sequential tasks. |
Context-Aware Cross-Lingual Mapping (N19-1)
Copied to clipboard
| Challenge: | Cross-lingual word vectors are typically obtained by fitting an orthogonal matrix that maps the entries of a bilingual dictionary from a source to a target vector space. |
| Approach: | They propose an alternative to word-level mapping that better reflects sentence-level cross-lingual similarity by directly mapping the averaged embeddings of aligned sentences in a parallel corpus. |
| Outcome: | The proposed approach outperforms context-independent word mapping in translation retrieval. |
Discriminative Reasoning with Sparse Event Representation for Document-level Event-Event Relation Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Document-level Event Causality Identification (DECI) is a sentence-level task that requires long-text understanding. |
| Approach: | They propose a document-level event causality identification model (SENDIR) that uses sparse attention to capture long-distance dependence. |
| Outcome: | The proposed model can be used to discriminate between event pairs in the same sentence or span multiple sentences. |